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ABSTRACT 


The  purpose  of  the  TIMS  program  (Phases  I  and  II)  was  to  design,  develop,  and 
evaluate  a  prototype  Training  Information  Management  System  (TIMS)  containing 
two  major  subsystems:  (1)  the  Electronic  Clipboard  Subsystem  (ECS),  a  hand-held 
device  which  contains  and  displays  performance  evaluation  checklists  and  other 
associated  information  to  allow  training  evaluators  to  record  the  success  or  failure 
of  soldiers  in  meeting  the  standards  of  performance  for  selected  tasks;  (2)  the 
Training  Base  Station  (TBS),  which  maintains  multiple  checklist  databases, 
transfers  data  to/from  the  ECS,  and  generates  summaries  of  training  performance. 

The  goal  of  both  program  phases  was  to  rapidly  produce  prototype  systems  that 
could  be  "touched,  handled,  and  kicked"  so  that  early  and  efficient  hands-on 
evaluation  of  the  device  by  the  user  community  would  be  possible. 

The  result  of  Phase  I  was  a  Brassboard  ECS  and  prototype  (VAX  1 1/70-based) 
TBS.  During  Phase  II,  twenty  ECS  field  units  and  an  updated  (IBM  PC-based)  TBS 
were  produced  and  field  tested  at  Fort  Knox,  Kentucky. 

Resulting  from  the  successful  field  test  were  a  set  of  user  recommendations  for 
improvements  that  should  be  included  in  the  next  version  of  TIMS. 


1 .0  PURPOSE  AND  SCOPE 


The  purpose  of  this  report  is  to  provide  a  technical  and  management  overview 
of  the  Phase  I  and  Phase  II  Technical  Information  Management  System  (TIMS) 
program.  This  report  contains  the  following  six  sections: 

1.  Purpose  and  scope 

2.  TIMS  background 

3.  Overview  of  technical  approach  and  "lessons  learned"  on 
Phase  I  program 

4.  Overview  of  technical  approach  and  "lessons  learned"  on 
Phase  II  program 

5.  Management  retrospective 

6.  Recommendations  for  future  TIMS  improvements 
2.0  TIMS  BACKGROUND 

The  Training  Information  Management  System  (TIMS)  is  a  computer-based 
system  that  can  be  used  by  Army  personnel  to  collect  and  display  training  evaluation 
data  during  field  training  exercises,  and  to  generate  summary  evaluation  reports 
following  the  training  exercises.  The  TIMS  has  two  major  subsystems.  The  first, 
called  the  Electronic  Clipboard  Subsystem  (ECS),  is  a  hand-held  electronic  field 
training  and  performance  evaluation  aid.  The  ECS  contains  and  displays 
performance  evaluation  checklists  and  other  associated  information  to  allow  a 
training  evaluator  to  record  the  success  or  failure  of  a  soldier  in  meeting  the 
standards  of  performance  for  selected  tasks. 

The  second  subsystem  is  called  the  Training  Base  Station  (TBS).  The  TBS  is  a 
computer-based  subsystem  that  maintains  multiple  checklist  databases,  transfers  data 
to/from  the  ECS,  and  generates  printed  and  displayed  summaries  of  training 
performance.  The  TBS  is  not  a  field-portable  subsystem,  but  rather  resides  at  a  fixed 
location  (e.g.,  the  unit  headquarters). 


The  purpose  of  the  TIMS  Phase  I  effort  was  to  develop  and  demonstrate  a 
preliminary  TIMS,  comprised  of  an  ECS  Brassboard  (ECSB)  and  a  prototype 
Training  Base  Station  Emulator  (TBSE).  The  purpose  of  the  Phase  II  effort  was  to 
build  upon  the  technical  information  and  expertise  gained  during  the  Phase  I  effort  to 
develop,  produce,  and  validate  an  Electronic  Clipboard  Subsystem  Prototype 
(ECSP),  and  then  fabricate,  field  test,  and  deliver  a  TIMS  comprised  of  twenty  (20) 
ECS  field  units  and  one  TBS. 

3.0  OVERVIEW  OF  TECHNICAL  APPROACH  AND  "LESSONS  LEARNED- 
ON  PHASE  I  PROGRAM 


The  Phase  I  effort  began  with  the  development  of  two  primary  documents, 
namely  (1)  the  Detailed  System  and  Subsystem  Requirements  document,  and  (2)  the 
TIMS  Functional  Specification.  These  two  documents  taken  together  described  to 
the  hardware  and  software  engineers  the  devices  that  were  to  be  produced.  The 
TIMS  Requirements  Document  focused  on  (1)  the  assumptions  about  the  use  of 
TIMS,  (2)  the  hardware  and  software  requirements,  (3)  the  operational 
requirements,  and  (4)  the  environmental  requirements  for  the  overall  system  and  the 
two  major  subsystems:  the  Electronic  Clipboard  Subsystem  (ECS)  and  the  Training 
Base  Station  Subsystem  (TBS). 

The  TIMS  Functional  Specification  was  based  on  an  analysis  of  (1)  the  training 
checklists  that  were  to  be  displayed  on  the  ECS  and  (2)  die  training  data  that  would  be 
collected  on  the  ECS  and  analyzed  at  the  TBS.  Whereas  the  TIMS  Requirements 
Document  gave  consideration  to  fabrication  concerns,  such  as  size,  weight,  power 
consumption,  etc.,  the  primary  focus  of  the  TIMS  Functional  Specification  was  on 
the  functional  characteristics  to  be  included  in  the  system.  Thus  a  major  section 
defined  the  functional  requirements  for  the  overall  Training  Information 
Management  System  (TIMS),  and  specifically  the  functional  characteristics  tint  were 
to  be  included  in  the  ECS  and  the  TBS.  These  functional  characteristics  were 
described  in  terms  of  the  display  screens  that  were  to  be  presented  to  the  ECS  and 
TBS  users,  and  the  controls  that  they  would  have  available  to  operate  these 
subsystems. 


Concurrent  with  the  development  of  the  TIMS  Requirements  Document  and 
the  TIMS  Functional  Specification,  the  hardware  engineers  began  a  preliminary 
analysis  of  the  components  that  would  meet  the  overall  needs  of  the  various 
subsystems.  It  was  recognized  immediately  that  the  design  of  the  ECS  would  be 
subject  to  severe  limitations  in  (1)  size  and  weight  imposed  by  the  need  for 
portability,  (2)  numbers  of  displayed  characters  imposed  by  the  need  for  large 
amounts  of  data  to  be  presented  at  any  one  time,  and  (3)  power  consumption  imposed 
by  the  need  for  extended  operations  between  recharging.  Thus  the  hardware 
engineers  focused  immediately  on  the  appropriate  display  technologies,  including 
Liquid  Crystal  Displays  (LCDs),  and  Electroluminescent  (EL)  panels.  They  also 
gave  immediate  attention  to  various  battery  alternatives,  and  die  available  low  power 
memory  technologies. 

As  soon  as  die  TIMS  Requirements  Document  was  approved,  and  the  basic 
hardware  technologies  (i.e.,  display,  battery,  and  memory)  had  been  selected,  the 
hardware  and  software  engineers  began  development  of  die  ECSB  and  TBSE.  The 
focus  of  the  Phase  I  effort  was  to  develop  a  Brassboard  ECS,  which  could  be  used  to 
present  a  concept  demonstration  of  the  Electronic  Clipboard.  Thus  the  focus  of  the 
hardware  and  software  development  was  to  produce  a  device  that  could  be  "touched, 
handled,  and  kicked,"  as  soon  as  possible  in  the  program.  Therefore,  the  effort  was 
not  focused  on  creating  a  fully  finished,  field-ready  device  that  contained  all  of  the 
possible  optimization  in  terms  of  hardware  design  (e.g.,  minimum  number  of  ICs, 
lowest  power  consumption,  etc.)  or  software  design  (e.g.,  modular  device  drivers, 
etc.). 

The  final  product  of  the  Phase  I  effort  was  an  ECS  Brassboard,  which 
consisted  of  a  hand-held  portion  that  contained  the  Liquid  Crystal  display,  attached 
by  cord  to  a  box  that  contained  the  wire-wrapped  PC  boards.  The  hand-held  portion 
of  the  ECSB  was  built  to  the  same  dimensions  (i.e.,  size  and  conformation)  as  the 
planned  ECS  Prototype,  and  weight  to  simulate  the  ECS  Prototype.  The  Training 
Base  Station  Emulator  software  operated  on  a  DEC  VAX  1 1/70,  and  was  connected 
by  a  serial  interface  to  the  TBSE  for  uploading  and  downloading  ECS  data  and 
programs. 


The  primary  technical  lesson  to  be  learned,  based  on  the  changes  that  occurred 
in  the  design  of  the  ECS  from  Phase  I  to  Phase  II,  is  to  be  sensitive  to,  and 
accommodate,  the  rapid  changes  that  are  occurring  in  electronics  technology.  For 
example,  in  the  short  time  (6  months)  between  selecting  the  LCD  for  the  Phase  I  ECS 
Brassboard  and  die  Phase  II  Prototype,  massive  and  significant  improvements 
occurred  in  LCD  technology.  These  advances  impacted  not  only  the  choice  of 
display,  but  also  the  display  drivers  and  the  case  size.  Similar  advances  can  be 
expected  in  the  development  of  any  ECS  and  TBS  that  will  be  developed  following 
Phase  II.  The  conclusion  to  be  drawn  is  (1)  to  expect  that  any  design,  be  it 
mechanical,  electrical,  or  software,  will  be  subject  to  significant  changes,  based  on 
advances  in  technology,  and  (2)  to  allow  for  such  advances  by  (a)  designing  sufficient 
flexibility  into  the  initial  design  to  accomodate  anticipated  advances  [note  that  such 
flexibility  may  be  inordinately  expensive  in  terms  of  schedule  and/or  budget],  or  (b) 
being  ready  to  make  changes  as  they  are  needed  or  possible. 

Another  important  lesson  concerns  the  amount  of  user  input  solicited  at  the 
end  of  Phase  I.  It  would  have  been  appropriate  to  invite  potential  end  users  to 
participate  in  the  Phase  I  Final  Review  and  to  "try  out"  the  ECS  Brassboard.  Given 
the  users'  comments  from  this  initial  "try  out,"  the  configuration  of  the  ECS  that 
would  have  been  developed  during  Phase  II  would  have  fit  more  closely  to  the  actual 
field  users'  expectations,  and  the  field  evaluation  conducted  at  the  end  of  Phase  II 
could  have  focused  more  extensively  on  validation  of  the  users'  requirements. 

4.0  OVERVIEW  OF  TECHNICAL  APPROACH  AND  "LESSONS  LEARNED- 

ON  PHASE  II  PROGRAM 

4.1  Phase  II  Technical  Approach 

The  Phase  II  effort  began  with  a  revision  of  the  TIMS  Functional  Specification 
developed  on  the  Phase  I  program.  The  purpose  of  this  specification  revision  was 
twofold:  (1)  to  update  the  system  requirements  in  light  of  the  technical  lessons 
learned  during  the  Phase  I  effort,  and  (2)  to  incorporate  new  requirements  stated  by 
the  Fort  Knox  Armor  School  representatives  during  a  meeting  on  1 1  July  1985. 
(Note:  the  major  new  requirement  was  for  a  capability  for  "three-pass  evaluation," 
whereby  students  are  given  up  to  three  opportunities  to  "pass"  a  given  checklist 
evalution  item).  A  new  requirements  document  was  published,  entitled  the 
"Functional  Specifications  for  the  Training  Information  Management  System 
(TIMS):  Phase  II". 


Development  of  an  ECS  Prototype  (ECSP)  began  in  earnest  after  the 
publication  of  the  new  requirements  document.  The  ECSP  hardware  and  software 
design  efforts  were  accompanied  by  the  development  of  initial  design 
documentation,  and  by  a  parallel  effort  for  revision  of  the  TBSE  design. 

As  the  designs  progressed,  work  began  on  the  development  of  a  test  plan  for 
validation  of  the  functionality  of  the  ECSP  and  TBS.  More  specifically,  both  a  Phase 
II  TIMS  Validation  Test  Plan  and  an  Electronic  Clipboard  System  Prototype  (ECSP) 
Validation  Test  Plan  were  developed.  Both  contained  actual  evaluation  checklists  to 
be  used  to  validate  the  conformance  of  the  TIMS  and  ECSP  with  the  physical, 
functional,  environmental,  and  interface  requirements  stated  in  the  Phase  II  TIMS 
requirements  document 

Also  prepared  during  the  design  phase  was  a  Phase  II  TIMS  Utilization  Plan 
document  which  defmed  the  requirements  for  the  Ft.  Knox  field  test  of  the  prototype 
TIMS.  This  document  contained  two  major  parts.  Part  I  was  the  Phase  II  TIMS 
Familiarization  and  Training  Plan,  which  outlined  the  approach  for  familiarizing 
and  training  U.S.  Army  personnel  at  Fort  Knox,  Kentucky,  in  the  use  of  the  Phase  II 
TIMS.  The  plan  outlined  the  requirements  for  both  classroom  and  "hands  on" 
training.  Part  II  was  the  Phase  II  TIMS  Field  Operation  Plan,  which  specified  the 
approach  for  generating,  collecting,  verifying,  storing,  and  disseminating 
appropriate  U.S.  Army  evaluator,  personel,  and  checklist  data  required  for 
evaluation  of  the  Phase  II  TIMS. 

Prototype  development  of  both  the  Phase  II  ECSP  and  the  TBS  was  completed 
at  approximately  the  same  time,  and  functional  testing  of  both  units  was  conducted 
according  to  the  pre-prepared  plans.  A  critical  finding  of  the  validation  testing  was 
that  the  power  requirements  of  certain  major  ECSP  components  was  far  greater  than 
anticipated,  thereby  causing  an  unacceptable  decrease  in  score  memory  retention 
time  and  necessitating  a  redesign.  By  far  the  greatest  part  of  the  problem  was  caused 
by  the  liquid  crystal  display,  which  was  a  recently  released  state-of-the-art  product 
for  which  only  preliminary  technical  data  was  available.  The  technical  data  provided 
by  the  vendor  proved  to  be  inaccurate  with  respect  to  the  power  requirements  of  the 
display.  This  led  to  the  power  problem  observed  during  die  validation  testing. 
Redesigned  circuitry,  allowing  for  a  complete  shut-down  of  the  display  during  the 
ECSP  power-down  mode,  was  developed  for  inclusion  in  the  20  field  units.  (Note:  a 
more  comprehensive  display  power-down  capabilitiy  had  not  been  included  in  the 
original  design  due  to  the  cost  and  space  requirements  associated  with  the  circuitry.) 


Subsequent  to  the  validation  tests,  and  during  the  time  that  development  of  the 
twenty  ECS  Field  Units  (incorporating  the  redesigned  circuitry)  was  underway, 
documentation  was  prepared  for  the  Field  Test.  Specifically,  the  following 
documents  were  developed: 


Phase  II  TIMS  Utilization  Documentation  -  which  presented  a  detailed 
description  of  the  training  activities  to  be  conducted  at  Fort  Knox,  and 
contained  the  actual  training  materials  to  be  presented 


Phase  II  TIMS  Evaluation  Plan  -  which  provided  a  description  of  the 
evaluation  and  data  analysis  methods  to  be  employed  during  the  field  test, 
and  included  the  actual  "structured  questionnaire"  form  which  was  to  be 
used  during  the  evaluation 


As  the  20  ECS  Field  Units  were  constructed,  the  functionality  of  these  units 
was  tested  in  accordance  with  the  requirements  of  a  pre-prepared  ECS  Field  Unit 
Acceptance  Test  Requirements  Document. 

The  results  of  all  functional  tests  (i.e.,  for  the  ECSP,  TIMS,  and  Field  Units) 
was  documented  in  deliverable  reports  (ECSP  Validation  Test  Documentation;  Phase 
II  TIMS  Validation  Test  Documentation;  ECS  Field  Units  Acceptance  Test  Results). 

The  Fort  Knox  field  test  was  conducted  during  the  month  of  November  1985. 
The  TIMS  evaluated  at  Fort  Knox  consisted  of  one  TBS  and  20  ECS  Field  Units.  At 
the  completion  of  the  field  test,  results  were  analyzed  and  then  summarized  in  a 
Phase  II  TIMS  Evaluation  Report.  A  set  of  recommendations  for  future  TIMS 
improvements,  based  upon  results  obtained  from  the  field  test  (and  documented  in 
the  Evaluation  Report),  was  developed  and  is  included  in  the  present  document  in 
section  6.0. 

Final  hardware  and  software  documentation  were  prepared  (i.e..  Final  ECSP 
Layout  and  Packaging  Drawings;  Final  Phase  II  ECSP  Software/Firmware  Design 
Documentation;  Final  TBS  Software  Design  Documentation)  to  provide  a 
permanent  record  of  the  final  design  configuration. 


User  inputs  regarding  the  TIMS  current  hardware  and  software  capabilities 
and  recommendations  for  both  immediate  and  long-term  product  improvement  were 
obtained  during  the  Fort  Knox  field  test  These  data  constitute  perhaps  the  best 
summary  of  "lessons  learned" and  is  summarized  in  a  document  entitled  Phase  II 
TIMS  Evaluation  Report 

5.0  MANAGEMENT  RETROSPECTIVE 


As  during  any  program  there  were  lessons  to  be  learned  from  a  management 
perspective.  In  this  section  four  specific  "lessons"  will  be  discussed  in  the  hopes  that 
they  may  benefit  future  programs. 

5.1  LESSON  1:  Requirements  Are  Bound  To  Change  So  BE  FLEXIBLE 
This  lesson  was  brought  home  in  three  separate  ways  during  the  program. 

(1)  An  early  design  decision  to  use  flexible  word  processing  and  data 
post-processing  programs  to  input  evaluation  checklist  databases  proved 
critical  in  allowing  for  easy  adaptation  to  an  unanticipated  change  in  the 
format  of  the  databases  required  to  be  used  for  the  Phase  II  evaluation. 

(2)  It  was  determined  during  the  conduct  of  the  program  that  additional 
flexibility  in  hardware  and  software  design ,  over  and  above  that 
incorporated  in  the  current  design,  is  desirable  to  accommodate  changes  in 
requirements,  (e.g.,  those  changes  that  occured  as  a  result  of  a  meeting  with 
the  Armor  community  at  the  beginning  of  Phase  II,  where  a  requirement 
for  "three  pass  scoring"  was  first  raised).  A  desirable  step  in  this  direction 
could  be  made  in  future  versions  of  TIMS  by  incorporating  a  capability  for 
downloading  not  only  databases,  but  also  applications  programs  into  RAM 
thereby  allowing  easy  adaptation  of  the  system  for  multiple  environments 
(without  the  necessity  of  burning  new  PROMS). 


(3)  Preparations  for  the  Fort  Knox  Field  test  were  extensive  and  included  the 
development  of  structured  questionnares  to  be  used  for  the  assessment  of 
users’  opinions.  These  questionnaires  were,  in  fact,  used  during  the  field 
evaluation  to  obtain  feedback  from  30  respondents.  However,  due  to  the 
tight  time  schedules  of  the  BNCOC  evaluators,  it  was  found  more  expedient 
to  extract  evaluation  data  from  them  in  a  more  free-form  format,  by  the  use 
of  verbal  questions  (modeled  after  questions  on  the  questionnaire)  and  "role 
playing",  whereby  the  evaluators  ran  through  scenarios  for  using  the 
Clipboard  under  operational  conditions  and  reported  their  thoughts  and 
impressions  as  to  the  strengths  and  weaknesses  of  the  current  design.  The 
lesson  here  is  simple  -  know  exactly  what  information  you  have  to  obtain, 
but  be  flexible  enough  to  obtain  that  information  in  various  formats. 


LESSON  2: 
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Both  ARI  and  JPL  participated  actively  on  the  program,  especially  during  the 
critical  parts  of  Phase  II,  the  TIMS  validation  testing  and  the  Fort  Knox  field 
evaluation.  Such  participation  allows  for  not  only  an  efficient  development  and 
review  process,  but  also  a  superior  product. 


LESSON  3: 
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A  prime  example  of  this  principle  concerns  the  development  of  the  ECSP  in 
Phase  II.  The  prototype  was  developed  for  purposes  of  assessing  the  functionality  of 
the  design  in  meeting  the  requirements  as  specified  in  the  Phase  II  requirements 
document  During  the  validation  testing,  design  changes  were  determined  to  be 
necessary  (for  example,  to  accommodate  the  power-saver  circuit).  However,  it  was 
determined  to  be  neither  efficient  nor  cost-effective  to  try  to  retrofit  the  existing  unit 
with  the  new  circuitry.  The  practical  approach  was  to  insure  that  all  required 
changes  were  incorporated  in  the  subsequent  20  field  units.  It  appears  that  in  cases 
such  as  this  it  is  not  cost-effective  to  require  delivery  of  development  models,  as  they 
are  suboptimal  in  comparison  to  the  final  units  and  contain  expensive  parts  which  can 
be  re-used  in  the  building  of  the  final  production  models. 
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5.4  LESSON  4:  Tw 


o  phases  of  field  testing  are  optimal. 

The  philosophy  used  during  the  current  contract  was  that  of  "rapid 
prototyping",  whereby  a  preliminary  product  is  produced  in  a  relatively  short 
period  of  time  so  that  a  product  rather  than  a  paper  specification,  can  be  evaluated 
early  by  the  customer  community.  Maximum  exploitation  of  such  an  approach 
requires  that  the  results  of  the  user  evaluation  be  analysed  and  the  critical 
recommendations  be  implemented  in  a  second  prototype,  which  is  then  returned  to 
the  field  for  a  final  evaluation.  Unfortunately,  only  a  single  evaluation  phase  was 
provided  for  in  the  current  contract. 

6.0  RECOMMENDATIONS  FOR  FUTURE  SYSTEM  IMPROVEMENTS 

As  a  result  of  the  analysis  of  the  field  evaluation  data,  a  set  of 
recommendations  was  developed  for  future  TIMS  improvements.  These 
recommendations  arc  documented  in  the  Phase  II  TIMS  Evaluation  Report  and  are 
also  included  herein  for  the  reader’s  convenience. 

The  system  features  recommended  for  inclusion  in  future  TIMS  designs  were 
divided  into  two  categories: 

•  Those  considered  to  be  very  important  to  include  in  a  new  design 

•  Those  important  for  future  consideration  but  optional  for 
the  next  generation  design  (i.e.,  not  absolutely  essential) 

6.1  Essential  Features  to  Include  in  Future.  Designs 

•  High  contrast  display 

•  Positive  feedback  on  touch  panel  (so  user  can  tell  when  keys  have  been 
pressed) 

•  Addition  of  a  Keyboard  Enable  button  that  tells  the  system  when  to  accept 
keyboard  inputs  (prevents  accidental  keypresses). 

•  Increased  ruggedization 


Faster  scroll 


Link  "Train  on  Drill"  and  "Evaluate  a  Drill"  modes  in  such  a  way  that  user 
can  go  rapidly  back  and  forth  between  them  without  losing  his  place. 


Do  not  automatically  route  user  into  next  unscored  pass  each  time  "Train 
On  Drill"  mode  is  requested.  Make  the  solider  indicate  that  he  has 
completed  a  pass  before  he  can  access  the  next  pass. 

Improve  the  means  of  indicating  when  a  given  key  selection  has  been  made 
(e.g.,  use  flashing  of  entire  key  selected,  or  inverse  video) 

Show  student  name  and  pass  number  at  all  times  in  the  "Evaluate  a  Drill" 
mode 

Add  an  "OFF'  switch  capability  for  the  backlight  and  power  (e.g.,  toggle 
the  light  and  power  switches). 

itional  Features  for  Next  Generation  Design 

Increase  Clipboard  memory  retention  time  from  five  days  to  over  one 
month 

Provide  intensity  control  for  electroluminescent  backlight 
Make  batteries  field  replaceable 

Increase  Clipboard  memory  so  Clipboard  can  store  multiple  training 
guides  simultaneously 

Provide  a  capability  to  charge  Clipboard  from  a  DC  power  source 
Add  a  numeric  keyboard  (software  keys) 

Make  displayed  error  messages  more  obvious 

Add  a  "set  marker"  function  that  will  allow  users  to  define  their  own  places 
in  the  database  to  which  they  can  rapidly  jump 
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In  addition  to  the  above,  the  following  recommendations  for  improvements 
were  made  by  the  developers  (Perceptronics): 

•  Consider  adding  a  capability  to  download  not  only  databases  but  also 
applications  programs  to  the  Clipboard.  This  will  allow  the  Clipboard  to 
be  easily  configured  to  support  multiple  requirements. 

•  Consider  rehosting  the  Training  Base  Station  onto  other  computers 
currently  in  use  by  major  user  communities. 


S«SW5 


E/VD 


9 


87 


DTIC 


